The present invention relates to subpixel rendering and gamut mapping in display devices.
Novel sub-pixel arrangements are disclosed for improving the cost/performance curves for image display devices in the following commonly owned United States Patents and Patent Applications including: (1) U.S. Pat. No. 6,903,754 (“the '754 Patent”) entitled “ARRANGEMENT OF COLOR PIXELS FOR FULL COLOR IMAGING DEVICES WITH SIMPLIFIED ADDRESSING;” (2) United States Patent Publication No. 2003/0128225 (“the '225 application”) having application Ser. No. 10/278,353 and entitled “IMPROVEMENTS TO COLOR FLAT PANEL DISPLAY SUB-PIXEL ARRANGEMENTS AND LAYOUTS FOR SUB-PIXEL RENDERING WITH INCREASED MODULATION TRANSFER FUNCTION RESPONSE,” filed Oct. 22, 2002; (3) United States Patent Publication No. 2003/0128179 (“the '179 application”) having application Ser. No. 10/278,352 and entitled “IMPROVEMENTS TO COLOR FLAT PANEL DISPLAY SUB-PIXEL ARRANGEMENTS AND LAYOUTS FOR SUB-PIXEL RENDERING WITH SPLIT BLUE SUB-PIXELS,” filed Oct. 22, 2002; (4) United States Patent Publication No. 2004/0051724 (“the '724 application”) having application Ser. No. 10/243,094 and entitled “IMPROVED FOUR COLOR ARRANGEMENTS AND EMITTERS FOR SUB-PIXEL RENDERING,” filed Sep. 13, 2002; (5) United States Patent Publication No. 2003/0117423 (“the '423 application”) having application Ser. No. 10/278,328 and entitled “IMPROVEMENTS TO COLOR FLAT PANEL DISPLAY SUB-PIXEL ARRANGEMENTS AND LAYOUTS WITH REDUCED BLUE LUMINANCE WELL VISIBILITY,” filed Oct. 22, 2002; (6) United States Patent Publication No. 2003/0090581 (“the '581 application”) having application Ser. No. 10/278,393 and entitled “COLOR DISPLAY HAVING HORIZONTAL SUB-PIXEL ARRANGEMENTS AND LAYOUTS,” filed Oct. 22, 2002; and (7) United States Patent Publication No. 2004/0080479 (“the '479 application”) having application Ser. No. 10/347,001 and entitled “IMPROVED SUB-PIXEL ARRANGEMENTS FOR STRIPED DISPLAYS AND METHODS AND SYSTEMS FOR SUB-PIXEL RENDERING SAME,” filed Jan. 16, 2003. Each of the aforementioned '225, '179, '724, '423, '581, and '479 published applications and U.S. Pat. No. 6,903,754 are hereby incorporated by reference herein in its entirety.
For certain subpixel repeating groups having an even number of subpixels in a horizontal direction, systems and techniques to affect improvements, e.g. polarity inversion schemes and other improvements, are disclosed in the following commonly owned United States patent documents: (1) United States Patent Publication No. 2004/0246280 (“the '280 application”) having application Ser. No. 10/456,839 and entitled “IMAGE DEGRADATION CORRECTION IN NOVEL LIQUID CRYSTAL DISPLAYS”; (2) United States Patent Publication No. 2004/0246213 (“the '213 application”) (U.S. patent application Ser. No. 10/455,925) entitled “DISPLAY PANEL HAVING CROSSOVER CONNECTIONS EFFECTING DOT INVERSION”; (3) United States Patent Publication No. 2004/0246381 (“the '381 application”) having application Ser. No. 10/455,931 and entitled “SYSTEM AND METHOD OF PERFORMING DOT INVERSION WITH STANDARD DRIVERS AND BACKPLANE ON NOVEL DISPLAY PANEL LAYOUTS”; (4) United States Patent Publication No. 2004/0246278 (“the '278 application”) having application Ser. No. 10/455,927 and entitled “SYSTEM AND METHOD FOR COMPENSATING FOR VISUAL EFFECTS UPON PANELS HAVING FIXED PATTERN NOISE WITH REDUCED QUANTIZATION ERROR”; (5) United States Patent Publication No. 2004/0246279 (“the '279 application”) having application Ser. No. 10/456,806 entitled “DOT INVERSION ON NOVEL DISPLAY PANEL LAYOUTS WITH EXTRA DRIVERS”; (6) United States Patent Publication No. 2004/0246404 (“the '404 application”) having application Ser. No. 10/456,838 and entitled “LIQUID CRYSTAL DISPLAY BACKPLANE LAYOUTS AND ADDRESSING FOR NON-STANDARD SUBPIXEL ARRANGEMENTS”; (7) United States Patent Publication No. 2005/0083277 (“the '277 application”) having application Ser. No. 10/696,236 entitled “IMAGE DEGRADATION CORRECTION IN NOVEL LIQUID CRYSTAL DISPLAYS WITH SPLIT BLUE SUBPIXELS”, filed Oct. 28, 2003; and (8) United States Patent Publication No. 2005/0212741 (“the '741 application”) having application Ser. No. 10/807,604 and entitled “IMPROVED TRANSISTOR BACKPLANES FOR LIQUID CRYSTAL DISPLAYS COMPRISING DIFFERENT SIZED SUBPIXELS”, filed Mar. 23, 2004. Each of the aforementioned '280, '213, '381, '278, '404, '277 and '741 published applications are hereby incorporated by reference herein in its entirety.
These improvements are particularly pronounced when coupled with sub-pixel rendering (SPR) systems and methods further disclosed in the above-referenced U.S. Patent documents and in commonly owned United States Patents and Patent Applications: (1) United States Patent Publication No. 2003/0034992 (“the '992 application”) having application Ser. No. 10/051,612 and entitled “CONVERSION OF A SUB-PIXEL FORMAT DATA TO ANOTHER SUB-PIXEL DATA FORMAT,” filed Jan. 16, 2002; (2) United States Patent Publication No. 2003/0103058 (“the '058 application”) having application Ser. No. 10/150,355 entitled “METHODS AND SYSTEMS FOR SUB-PIXEL RENDERING WITH GAMMA ADJUSTMENT,” filed May 17, 2002; (3) United States Patent Publication No. 2003/0085906 (“the '906 application”) having application Ser. No. 10/215,843 and entitled “METHODS AND SYSTEMS FOR SUB-PIXEL RENDERING WITH ADAPTIVE FILTERING,” filed Aug. 8, 2002; (4) United States Publication No. 2004/0196302 (“the '302 application”) having application Ser. No. 10/379,767 and entitled “SYSTEMS AND METHODS FOR TEMPORAL SUB-PIXEL RENDERING OF IMAGE DATA” filed Mar. 4, 2003; (5) United States Patent Publication No. 2004/0174380 (“the '380 application”) having application Ser. No. 10/379,765 and entitled “SYSTEMS AND METHODS FOR MOTION ADAPTIVE FILTERING,” filed Mar. 4, 2003; (6) U.S. Pat. No. 6,917,368 (“the '368 Patent”) entitled “SUB-PIXEL RENDERING SYSTEM AND METHOD FOR IMPROVED DISPLAY VIEWING ANGLES”; and (7) United States Patent Publication No. 2004/0196297 (“the '297 application”) having application Ser. No. 10/409,413 and entitled “IMAGE DATA SET WITH EMBEDDED PRE-SUBPIXEL RENDERED IMAGE” filed Apr. 7, 2003. Each of the aforementioned '992, '058, '906, '302, 380 and '297 applications and the '368 patent are hereby incorporated by reference herein in its entirety.
Improvements in gamut conversion and mapping are disclosed in commonly owned United States Patents and co-pending United States Patent Applications: (1) U.S. Pat. No. 6,980,219 (“the '219 Patent”) entitled “HUE ANGLE CALCULATION SYSTEM AND METHODS”; (2) United States Patent Publication No. 2005/0083341 (“the '341 application”) having application Ser. No. 10/691,377 and entitled “METHOD AND APPARATUS FOR CONVERTING FROM SOURCE COLOR SPACE TO TARGET COLOR SPACE”, filed Oct. 21, 2003; (3) United States Patent Publication No. 2005/0083352 (“the '352 application”) having application Ser. No. 10/691,396 and entitled “METHOD AND APPARATUS FOR CONVERTING FROM A SOURCE COLOR SPACE TO A TARGET COLOR SPACE”, filed Oct. 21, 2003; and (4) United States Patent Publication No. 2005/0083344 (“the '344 application”) having application Ser. No. 10/690,716 and entitled “GAMUT CONVERSION SYSTEM AND METHODS” filed Oct. 21, 2003. Each of the aforementioned '341, '352 and '344 applications and the '219 patent is hereby incorporated by reference herein in its entirety.
Additional advantages have been described in (1) United States Patent Publication No. 2005/0099540 (“the '540 application”) having application Ser. No. 10/696,235 and entitled “DISPLAY SYSTEM HAVING IMPROVED MULTIPLE MODES FOR DISPLAYING IMAGE DATA FROM MULTIPLE INPUT SOURCE FORMATS”, filed Oct. 28, 2003; and in (2) United States Patent Publication No. 2005/0088385 (“the '385 application”) having application Ser. No. 10/696,026 and entitled “SYSTEM AND METHOD FOR PERFORMING IMAGE RECONSTRUCTION AND SUBPIXEL RENDERING TO EFFECT SCALING FOR MULTI-MODE DISPLAY” filed Oct. 28, 2003, each of which is hereby incorporated herein by reference in its entirety.
Additionally, each of these co-owned and co-pending applications is herein incorporated by reference in its entirety: (1) United States Patent Publication No. 2005/0225548 (“the '548 application”) having application Ser. No. 10/821,387 and entitled “SYSTEM AND METHOD FOR IMPROVING SUB-PIXEL RENDERING OF IMAGE DATA IN NON-STRIPED DISPLAY SYSTEMS”; (2) United States Patent Publication No. 2005/0225561 (“the '561 application”) having application Ser. No. 10/821,386 and entitled “SYSTEMS AND METHODS FOR SELECTING A WHITE POINT FOR IMAGE DISPLAYS”; (3) United States Patent Publication No. 2005/0225574 (“the '574 application”) and United States Patent Publication No. 2005/0225575 (“the '575 application”) having application Ser. Nos. 10/821,353 and 10/961,506 respectively, and both entitled “NOVEL SUBPIXEL LAYOUTS AND ARRANGEMENTS FOR HIGH BRIGHTNESS DISPLAYS”; (4) United States Patent Publication No. 2005/0225562 (“the '562 application”) having application Ser. No. 10/821,306 and entitled “SYSTEMS AND METHODS FOR IMPROVED GAMUT MAPPING FROM ONE IMAGE DATA SET TO ANOTHER”; (5) United States Patent Publication No. 2005/0225563 (“the '563 application”) having application Ser. No. 10/821,388 and entitled “IMPROVED SUBPIXEL RENDERING FILTERS FOR HIGH BRIGHTNESS SUBPIXEL LAYOUTS”; and (6) United States Patent Publication No. 2005/0276502 (“the '502 application”) having application Ser. No. 10/866,447 and entitled “INCREASING GAMMA ACCURACY IN QUANTIZED DISPLAY SYSTEMS.”
Additional improvements to, and embodiments of, display systems and methods of operation thereof are described in: (1) Patent Cooperation Treaty (PCT) Application No. PCT/US 06/12768, entitled “EFFICIENT MEMORY STRUCTURE FOR DISPLAY SYSTEM WITH NOVEL SUBPIXEL STRUCTURES” filed Apr. 4, 2006; (2) Patent Cooperation Treaty (PCT) Application No. PCT/US 06/12766, entitled “SYSTEMS AND METHODS FOR IMPLEMENTING LOW-COST GAMUT MAPPING ALGORITHMS” filed Apr. 4, 2006; (3) U.S. patent application Ser. No. 11/278,675, entitled “SYSTEMS AND METHODS FOR IMPLEMENTING IMPROVED GAMUT MAPPING ALGORITHMS” filed Apr. 4, 2006, and published as United States Patent Application Publication 2006/0244686; (4) Patent Cooperation Treaty (PCT) Application No. PCT/US 06/12521, entitled “PRE-SUBPIXEL RENDERED IMAGE PROCESSING IN DISPLAY SYSTEMS” filed Apr. 4, 2006; and (5) Patent Cooperation Treaty (PCT) Application No. PCT/US 06/19657, entitled “MULTIPRIMARY COLOR SUBPIXEL RENDERING WITH METAMERIC FILTERING” filed on May 19, 2006. Each of these co-owned applications is also herein incorporated by reference in its entirety.
As explained in some of the above patent applications, an image 104 (FIG. 1) is represented by a number of areas 106 (FIG. 1) called pixels. Each pixel 106 is associated with a color that should be substantially reproduced by a set of subpixels in a display 110. Conventionally, each subpixel displays a “primary” color, i.e. each conventional subpixel is associated with some hue and saturation. Other colors are obtained by mixing primary colors. Each pixel 106 is mapped into a set of one or more subpixels which are to display the pixel's color.
In some displays, each repeating set of subpixels includes a subpixel for each primary color. The subpixels are small, and are spaced closely together, to provide a desired resolution. This structure is not cost-effective however because it does not match the resolution of human vision. Humans are more perceptive to luminance differences than to chromatic differences. Therefore, some displays map an input pixel 106 into a subpixel repeating set that does not include only the subpixels of each primary color. The chromatic resolution is reduced, but the luminance resolution remains high.
One such display 110 is described in PCT application published as no. WO 2006/127555 A2 on 30 November 2006 and U.S. patent application Ser. No. 11/278,675 published as no. 2006/0244686 A1 on 2 Nov. 2006 and illustrated in FIG. 1. The display 110 is of RGBW type, with red subpixels 120R, blue subpixels 120B, green subpixels 120G, and white subpixels 120W. All these subpixels 120 are equal in area. Each set of subpixels consists of two adjacent subpixels in the same row. These sets 124 are called “pairs” below. Each pair 124 consists of either a red subpixel 120R and a green subpixel 120G (such pairs are called “RG pairs” below), or each pair consists of a blue subpixel 120B and a white subpixel 120W (“BW pair”). In each RG pair, the red subpixel is to the left of the green one. In each BW pair, the blue subpixel is on the left. The RG and BW pairs alternate in each row and each column.
Each pixel 106 in column x and row y of the image (pixel “106x,y” below) is mapped into the subpixel pair 124 in column x and row y (“124x,y” below). In display 110, the consecutive indices x and y denote consecutive pairs, not consecutive subpixels. Each pair 124 has only two subpixels, and provides a high range and resolution in luminance but not in chrominance. Therefore, part of the input pixel's luminance may have to be shifted to adjacent pairs 124 in a “subpixel rendering” operation (SPR) described in some of the aforementioned patent applications and illustrated in FIG. 2.
FIG. 2 illustrates the SPR operation for the red and green subpixels. The blue and white subpixels are treated in a similar manner. The SPR operation calculates the values Rw, Gw, Bw, Ww defining the luminances for the respective read, green, blue and white subpixels in a linear manner, i.e. the luminances are linear functions of the subpixel values (different functions may be used for different primary colors). The Rw, Gw, Bw, Ww values are then used to determine electrical signals provided to the subpixels to obtain the desired luminances.
FIG. 2 shows the pixels 106 superimposed onto the respective subpixel pairs 124. The blue and white subpixels are not shown. The display area is subdivided into “sampling” areas 250 centered at the respective RG pairs 124. The sampling areas 250 can be defined in different ways, and in FIG. 2 diamond-shaped areas 250 are chosen. The areas 250 are congruent to each other except at the edges of the display.
The color of each pixel 106 is expressed in a linear RGBW color coordinate system. For each RG pair 124x,y, the Rw value of the red subpixel is determined as a weighted sum of the R coordinates of all the pixels 106 which overlap with the sampling area 250 centered at the RG pair 124x,y. The weights are chosen to add up to 1, and are proportional to the areas of overlap of the respective pixels 106 with the sampling area 250. In particular, if the subpixel pair 124x,y is not at the edge of the display, then the red value Rw is:Rw=½*Rx,y+⅛*Rx−1,y+⅛*Rx+1,y+⅛*Rx,y−1+⅛*Rx,y+1  (1)
In other words, the red subpixels 120R can be rendered by applying a 3×3 diamond filter to the R coordinates of the respective pixels 106 with the following filter kernel:
                    (                                            0                                                      1                /                8                                                    0                                                                          1                /                8                                                                    1                /                2                                                                    1                /                8                                                                        0                                                      1                /                8                                                    0                                      )                            (        2        )            
The same filter kernel can be used for the green, blue and white subpixels (except at the edges). Other filter kernels can also be used. See e.g. the aforementioned United States Patent Publication No. 2005/0225563.
The luminance shifts performed in the sub-pixel rendering may undesirably cause image degradation such as blurriness or loss of local contrast. The image can be improved by applying sharpening filters (e.g. DOG, i.e. Difference of the Gausians). See e.g. the aforementioned PCT application WO 2006/127555. Additional improvements in image quality are desirable.
Further, some of the operations described above may cause some subpixel values to be out of gamut, especially if the gamut is restricted in brightness to reduce power consumption. Forcing the subpixel values into the available gamut may distort the image, e.g. reduce local contrast, and such distortion should be minimized. It is desirable to improve gamut mapping operations, especially in low brightness environments.